1. Field of the Invention
The present invention relates to an image pickup apparatus, a method for controlling an image pickup apparatus, a signal processing apparatus, and a method for processing a signal.
2. Description of the Related Art
In recent years, with the advancement of technology, the size of a solid-state image pickup element, such as a charged coupled device (CCD), and power consumption of a solid-state image pickup element have been reduced. In addition, a large scale integration (LSI) has been more highly integrated, an LSI has had a higher performance, and power consumption of an LSI has been reduced. Accordingly, mobile image pickup apparatuses (e.g., digital still cameras) that mainly capture still images and that can be battery operated have been in widespread use.
In such image pickup apparatuses that can be battery operated, in order to increase an operating time and, thus, maximize the number of shots that can be taken, it is desirable that power consumption is reduced.
For image pickup apparatuses, in general, as the frame rate and the resolution of a captured image increases, the amount of data of an image processed per unit time increased. Accordingly, it is necessary to provide a high clock frequency to a circuit that processes such image data.
However, the power consumption of a circuit increases in proportion with the clock frequency used for driving the circuit. Therefore, if the frame rate and the resolution of image data increase, the power consumption increases. Consequently, in order to reduce the power consumption of a circuit, reduction in the frame rate and the resolution is necessary.
However, for image pickup apparatuses (e.g., digital still cameras), the requirement for the frame rate and the resolution of a captured image vary in accordance with an operation mode. For example, in a monitoring mode in which the image of an object is displayed on a display unit (e.g., a monitor) provided on an image pickup apparatus and the angle of view is adjusted before the image is captured, it is desirable that a smoothly moving image is displayed on the display unit. In general, the display unit provided on the image pickup apparatus is small. Accordingly, it is desirable that the frame rate of the moving image is high. However, a high resolution of the image is not necessary.
In contrast, for example, in a shooting mode in which the images of an object are captured, in order to maximize the continuous shooting performance, it is desirable that the processing rate and the resolution of images are maximized. Furthermore, in a playback mode in which a captured still image is played back, an image signal output from an image pickup element is not used, and the resolution of the still image is maximized when the still image is displayed on a display unit.
Such an image pickup apparatus includes a variety of processing blocks, such as an image capturing block, an image processing block, and a display block. The amounts of data processed by these processing blocks per unit time are significantly different in accordance with the operation mode. That is, it is not necessary that these processing blocks operate at their maximum clock frequencies at all times. Therefore, the power consumption of the image pickup apparatus can be reduced by operating the processing blocks at their minimized clock frequencies that allow these processing blocks to normally operate.
Accordingly, existing image pickup apparatuses reduce power consumption by stopping supplying a clock signal to a processing block that is in a completely non-operating state or switching clock frequencies in accordance with a necessary processing rate for the current operation mode.
As shown in FIG. 14, an existing image pickup apparatus 100 receives light made incident on an optical system 110 using an image pickup element 111 and converts the light into an imaging signal. Subsequently, the image pickup apparatus 100 processes the imaging signal using a signal processing unit 120 and a random access memory (RAM) 136, which is an external memory connected to the signal processing unit 120. The imaging signal processed in a predetermined manner is stored in a recording medium 138. The image pickup apparatus 100 stores the imaging signal in the recording medium 138 in the form of moving image data. In addition, the image pickup apparatus 100 can store the imaging signal in the recording medium 138 in the form of still image data captured at an instructed time.
The optical system 110 includes a lens system, an aperture mechanism, a focusing mechanism, and a zooming mechanism. Under the control of a drive unit 113 based on an instruction received from a central processing unit (CPU) 131 (described below) and a manual operation, aperture control, focusing, and zooming are performed. Light output from an object is made incident on the image pickup element 111 via the optical system 110. The image pickup element 111 is formed from, for example, a CCD. The image pickup element 111 converts the incident light into an electrical signal and outputs the electrical signal in a line-sequential manner in the form of an imaging signal. It should be noted that the image pickup element 111 is not limited to a CCD. For example, the image pickup element 111 may be formed from a complementary metal-oxide semiconductor (CMOS) imager.
A front-end (F/E) unit 112 performs correlated double sampling processing on the imaging signal output from the image pickup element 111 in an analog format. In addition, the front-end unit 112 controls the gain of the imaging signal. Thereafter, the Front-end 112 converts the analog imaging signal into a digital imaging signal, which is raw data.
Using a timing signal (a clock pulse) output from a timing generator 114 described below, the image pickup element 111 is controlled so that photoelectric conversion is carried out and an imaging signal is output at predetermined frame intervals. In addition, the Front-end 112 is controlled so as to operate in synchronization with the frame intervals.
The signal processing unit 120 includes the following processing blocks: a sensor interface (I/F) 122, a detection correcting unit 123, a signal processing unit 124, a resolution conversion unit 125, an extended image processing unit 126, a still image codec unit 127, a moving image codec unit 128, a display control unit 129, the CPU 131, an external interface (I/F) 134, a memory controller 135, and a recording and playback control unit 137, which are connected with one another via a bus 121. For example, the signal processing unit 120 is in the form of one LSI.
The raw data output from the front-end unit 112 is input to the sensor I/F 122. The sensor I/F 122 changes the data sequence of the raw data so that the raw data has a data sequence optimal for a correction process and a color conversion process performed as downstream processing. In addition, the sensor I/F 122 switches the use of a sensor driving clock to the use of a system clock. That is, the data supplied to the sensor I/F 122 is the output of the image pickup element 111. The output of the image pickup element 111 is synchronized with a driving clock used for reading out electrical charge from the image pickup element 111. On the other hand, it is necessary that data output from the sensor I/F 122 be synchronized with the internal clock of the signal processing unit 120. Accordingly, the sensor I/F 122 includes a buffer memory. The sensor I/F 122 writes the received data into the buffer memory in synchronization with the sensor driving clock. Subsequently, the sensor I/F 122 reads out the data from the buffer memory in synchronization with the system clock.
The detection correcting unit 123 performs a variety of correction processes (e.g., a defect correction process and a lens correction process) on the supplied raw data. In addition, the detection correcting unit 123 performs a detection process for adjusting a black level, focus, exposure, white balance.
The signal processing unit 124 converts the raw data into an imaging signal having a format suitable for the downstream processing. For example, the signal processing unit 124 performs demosaic processing on the raw data so as to generate a Y/C signal including a luminance signal Y and a color difference signal Cb/Cr. In addition, the signal processing unit 124 performs predetermined processing, such as white balance processing, gamma correction processing, and unsharp mask processing.
In general, the image qualities desired for a still image and a moving image are different. Accordingly, under the control of the CPU 131, the signal processing unit 124 can switch between parameters used when the image correction processing is performed on still image data and when the image correction processing is performed on moving image data.
The resolution conversion unit 125 converts the resolution of baseband moving image data or still image data into a resolution suitable for a display unit 130 described below or a resolution suitable for a recording mode. As used herein, the term “baseband data” refers to data obtained by converting a signal of an analog format to a digital format before being subjected to various processing, such as compression encoding and modulation. In this example, a digital imaging signal obtained by converting an analog signal output from the image pickup element 111 into a digital signal by means of the front-end unit 112 is referred to as “baseband data” when necessary.
The extended image processing unit 126 performs extended image processing, such as an object recognition process or a super high-speed continuous shooting process. In the object recognition process, for example, an image of a human face is extracted from the entire image using a pre-learned pattern dictionary. In the super high-speed continuous shooting process, electrical charge in the image pickup element 111 is read out at a significantly high frame rate of, for example, 240 fps (frames per second) for a moving image, as compared with a normal frame rate of 60 fps. A high-speed data transfer capability is necessary for a super high-speed continuous shooting function, as compared with a normal moving image shooting function. Accordingly, in order to provide the super high-speed continuous shooting function, it is desirable that a dedicated processing unit is provided.
The still image codec unit 127 performs a compression encoding process on baseband still image data and a decoding process on compression-encoded still image data. Any type of compression encoding method can be employed. For example, a JPEG (joint photographic expert group) encoding method may be employed. The decoding process of the compressed still image data performed by the still image codec unit 127 is a process that is the reverse of the compression-encoding process.
The moving image codec unit 128 performs a compression-encoding process on baseband moving image data and a decoding process on compression-encoded moving image data. Any type of compression encoding method can be employed. For example, an MPEG 2 (Moving Picture Expert Group 2) encoding method may be employed. The decoding process of the compressed moving image data performed by the moving image codec unit 128 is a process that is the reverse of the compression-encoding process.
The display control unit 129 converts supplied image data into a signal having a format displayable on the display unit 130. For example, the display unit 130 is formed from a liquid crystal display (LCD). The display unit 130 is used for a viewfinder of the image pickup apparatus 100. The display unit 130 further functions as a monitor used for playing back an image stored in the recording medium 138.
A read only memory (ROM) 132 and an input device 133 are connected to the CPU 131. The CPU 131 uses a random access memory (RAM) (not shown) as a work memory so as to perform overall control of the image pickup apparatus 100 in accordance with the instructions of prestored program in the ROM 132.
For example, the CPU 131 communicates a command and data with the blocks of the signal processing unit 120 via the bus 121 so as to control the blocks of the signal processing unit 120. In addition, the CPU 131 generates control signals for controlling focusing, an aperture, and zooming of the optical system 110 on the basis of control signals and an imaging signal in accordance with the operations performed on the input device 133 and supplies the generated control signals to the drive unit 113. The drive unit 113 controls the blocks of the optical system 110 in accordance with the supplied control signals. Furthermore, the CPU 131 submits a command to the timing generator 114 so that the timing generator 114 outputs a predetermined clock pulse.
In response to the commands supplied from the CPU 131, the timing generator 114 generates timing signals necessary for processing the imaging signal output from the image pickup element 111. Examples of the timing signals include a frame synchronization signal, a horizontal synchronization signal, and a vertical synchronization signal.
The input device 133 includes a variety of operating members used for operating the image pickup apparatus 100. The input device 133 outputs a control signal corresponding to an operation performed on one of the operating members. For example, the input device 133 includes a power key used for switching on an off the power supplied from a power supply unit 140, a mode switch key used for switching between the operation modes of the image pickup apparatus 100, such as an image capturing mode and a recording mode, and a key used for moving a cursor. In addition, the input device 133 includes a shutter button used for capturing a still image in the image pickup apparatus 100 and an operating member used for changing focusing, an aperture, and zooming of the optical system 110 when a still image or a moving image is captured.
The external I/F 134 controls exchange of data between the image pickup apparatus 100 and an external apparatus.
The RAM 136 is connected to the memory controller 135. For example, the RAM 136 is formed from a synchronous dynamic random-access memory (SDRAM) operating in synchronization with a memory bus clock. The RAM 136 can perform data input and output with a predetermined data length (a burst length) using burst transfer. The RAM 136 is also used by the units connected to the bus 121. The memory controller 135 performs access control to the RAM 136. When a transfer request of data to the RAM 136 is issued from one of the processing blocks connected to the bus 121, the data transfer request is delivered from the bus 121 to the memory controller 135. The memory controller 135 performs the access control to the RAM 136 in response to the data transfer request.
The recording and playback control unit 137 performs control for recording data into the recording medium 138 and control for playing back data recorded in the recording medium 138. An example of the recording medium 138 is a removable nonvolatile memory.
For example, Japanese Unexamined Patent Application Publication No. 2001-238190 describes a technique for reducing power consumption of such an image pickup apparatus by controlling a clock signal and the voltage of a power supply supplied to circuits corresponding to the above-described function blocks in accordance with an operation mode.